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Abstract: A neural network is an artificial representation of the human brain that tries to siirnl^C^ts 
learning process. An artificial neural network (ANN) is often called a "Neural Network " or singly Veural 
Net (NN). This paper summarizes the some of the most important developments in neural r^tocm. and its 
learning techniques. Learning can be done in supervised or unsupervised training purpose is to 
provide a synthesis of the published research in this area and stimulate further i|gl^ar<*h interests and 
efforts in the identified topics. 



r _^3ta£cti 



Index Terms — Learning and learning process, costs, neural networks, Error _^^jrJPction learning, Memory 
based learning, mathematical ananlysis, Artificial intelligence. 

Introduction 

Research in the field of neural networks has been attracting inciMsW attention in recent years. Since 1943, 
when Warren McCulloch and Walter Pitts presented the fiiVt model of artificial neurons, new and more 
sophisticated proposals have been made from decade to AScre. Mathematical analysis has solved some of 
the mysteries posed by the new models but has left m^S^uestions open for future investigations. Needless 
to say, the study of neurons, their interconnection3K»atheir role as the brain's elementary building blocks 
is one of the most dynamic and important reseS^ch fields in modern computer science. It is not an 
exaggeration to say that we have learned rpEVbout the nervous system in the last fifty years than ever 
before. Artificial neural networks are aj^Jl^mpt at modeling the information processing capabilities of 
nervous system. t^t 



Neural networks are a branch iOT^Arfificial Intelligence". Artificial Neural Network is a system loosely 
modeled based on the hui|fa^brain. The field goes by many names, such as connectionism, parallel 
distributed processing, rA^TOmputing, natural intelligent systems, machine learning algorithms, and 
artificial neural netwof^^W^ural networks are a powerful technique to solve many real world problems. 
They have the abjWl^tb© learn from experience in order to improve their performance and to adapt 
themselves to J^Ag^ in the environment. In addition to that they are able to deal with incomplete 
information ffllLiolsy data and can be very effective especially in situations where it is not possible to define 
the rules ^jnpVps that lead to the solution of a problem. 

One of the most important aspects of Neural Network is the learning process. The learning process of a 
Neural Network can be viewed as reshaping a sheet of metal, which represents the output (range) of the 
function being mapped. The training set (domain) acts as energy required to bend the sheet of metal such 
that it passes through predefined points. However, the metal, by its nature, will resist such reshaping. So 
the network will attempt to find a low energy configuration (i.e. a flat/non-wrinkled shape) that satisfies 
the constraints (training data). 



Learning Process 
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Learning is a process by which the free parameters of a neural network are adapted through a process of 
stimulation by the environment in which the network is embedded. The type of learning is determined by 
the manner in which the parameter changes take place. 

This definition of the learning process implies the following sequence of events: 

1. The neural network is stimulated by an environment. 

2. The neural network undergoes changes in its free parameters as a result of this stimulation 

3. The neural network responds in a new way to the environment because of the changes t^a^^^e 



occurred in its internal structure. 

Learning can be done in supervised or unsupervised training. In supervised training, both tf^iitput: 
the outputs are provided. The network then processes the inputs and compares its resultiimoWputs against 
the desired outputs. Errors are then calculated, causing the system to adjust the wemp^wfcich control the 
network. This process occurs over and over as the weights are continually tweaked. C 

Supervised and un-supervised* 



Two possible solutions 







I^thimpSper we are going to learn different learning techniques those are 1. Error_ correction learning 2. 
MeS^y based learning 

1. Error_correction learning: To understand this technique, assume a simple case of a neuron k 
constituting the only computational node in the output layer of a feed forward neural network, as depicted 
in figi 
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Neuron k is driven by a signal vector x(n) produced by one or more layers of hidden ne^ofef which are 



Iden ne< 

themselves driven by an input vector applied to the source nodes of the neural netwod^C 
Consequently, an error signal, denoted by ek(n), is produced. By definition, ^^/^ 

e k (n) =d k (n)-y k (n) 
this objective is achieved by minimizing a cost function or index o 



the error signal e k (n) as 

£(n) = % e 2 k (n) 



.\Ty 

i^^Qlnance, £(n), defined in terms of 



A* 

fptilk ex( 



Let w k j(n) denote the value of synaptic weight w k j of neiflKxNfck excited by element Xj(n) of the signal vector 
x(n) at time step n. in effect , w k j (n) and w k j (n+i) n^^»e*viewed as the old new values of synaptic weight 
w k j, respectively. In computational terms we may^fe^/rite 

w kj (n)= z _1 [w kj (n+i)] 

where z 1 is the unit delay operator. 2Ttf fosjrepresents a storage element. 

2. Memory-based learning: ra^e air* the past experiences are explicitly stored in a large memory of 



correctly classified input- ou 

All memory-based learnifcSgorithms involve two essential ingredients: 



• Criteria 

• Learnr 



qn i^eXror defining the local neighborhood of the test vector X test . 
lio^Jh^pplied to the training examples in the local neighborhood of X t€ 

tsVre 



The algoritlmjL^re differ from differ from each other in the way in which these two ingredients are defined. 

Ivpmqaqp neighborhood of the test vector X test . In particular, the vector 

\r X'n G { X D X 2 , . . . , X N } 

Is said to be the nearest neighbor of X test if 
min d(Xi, X tes t) = d(X' N , X test ) 
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where d(X' N , X test ) is the Euclidean distance between the vectors X[ and X test . The class associated with the 
minimum distance that is vector X' N is reported as the classification of X test . this rule is independent of the 
underlying distribution responsible for generating the training examples. 

A variant of the nearest neighbor classifier is the k-nearest neighbor classifier which proceeds as follows: 

• Identify the k classified patterns that lie nearest to the test vector X test for some integer k. 

• Assign to the class that is most regularly represented in the k nearest neighbors to X test . 

Thus the k-nearest neighbor classifier acts like an averaging device. 

Conclusion 

In this paper, we have studied one of the most important limitations of nei^iSiL networks, that is 
understanding why an neural networks, that is understanding why an neural net\\^rVjnakes a particular 
decision is a very difficult task. Also we have learn about learning processes ai^^ow it works. How the 
data is divided into supervised and un-supervised. And what are the different^^^mg techniques and how 
those works under various neural networks. * 
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